Three-dimensional (3D) television has been a technology trend in recent years that intends to bring viewers sensational viewing experience. Various technologies have been developed to enable 3D viewing. Among them, the multi-view video is a key technology for 3D TV application among others. The traditional video is a two-dimensional (2D) medium that only provides viewers a single view of a scene from the perspective of the camera. However, the multi-view video is capable of offering arbitrary viewpoints of dynamic scenes and provides viewers the sensation of realism.
The multi-view video is typically created by capturing a scene using multiple cameras simultaneously, where the multiple cameras are properly located so that each camera captures the scene from one viewpoint. Accordingly, the multiple cameras will capture multiple video sequences corresponding to multiple views. In order to provide more views, more cameras have been used to generate multi-view video with a large number of video sequences associated with the views. Accordingly, the multi-view video will require a large storage space to store and/or a high bandwidth to transmit. Therefore, multi-view video coding techniques have been developed in the field to reduce the required storage space or the transmission bandwidth.
A straightforward approach may be to simply apply conventional video coding techniques to each single-view video sequence independently and disregard any correlation among different views. Such coding system would be very inefficient. In order to improve efficiency of multi-view video coding, multi-view video coding exploits inter-view redundancy. Various 3D coding tools have been developed or being developed by extending existing video coding standard. For example, there are standard development activities to extend H.264/AVC (advanced video coding) and HEVC (high efficiency video coding) to multi-view video coding (MVC) and 3D coding. The corresponding new standards being developed are referred as 3D-HEVC (High Efficiency Video Coding) or 3D-AVC (Advanced Video Coding) coding respectively. Various 3D coding tools developed or being developed for 3D-HEVC and 3D-AVC are reviewed as follows.
FIG. 1 illustrates an example of 3D video coding system incorporating Disparity-Compensated Prediction (DCP) and motion-compensated prediction (MCP). The vector (110) used for DCP is termed as disparity vector (DV), which is analog to the motion vector (MV) used in MCP. FIG. 1 illustrates three MVs (120, 130 and 140) associated with MCP. Moreover, the DV 110 of a DCP block can also be predicted by the disparity vector predictor (DVP) candidate derived from neighboring blocks or the temporal collocated blocks that also use inter-view reference pictures. The inter-view reference picture (160) may have the same picture order count (POC) as the current picture (150). However, the inter-view reference picture (160) has a difference view index from the current picture (150) since they are in different views.
In the current 3D-HEVC, inter-view motion prediction is used to share the previously encoded motion information of reference views. For deriving candidate motion parameters for a current block in a dependent view, a DV for the current block is derived first, and then the prediction block in the already coded picture in the reference view is located by adding the DV to the location of the current block. If the prediction block is coded using MCP, the associated motion parameters can be used as candidate motion parameters for the current block in the current view. The DV can also be directly used as a candidate DV for DCP.
For the current block, motion vector predictor (MVP)/disparity vector predictor (DVP) can be derived from the inter-view blocks in the inter-view pictures. In the following, inter-view blocks in inter-view picture may be abbreviated as inter-view blocks. The derived candidate is termed as inter-view candidates, which can be inter-view MVPs or DVPs. The coding tools that codes the motion information of a current block (e.g., a current prediction unit, PU) based on previously coded motion information in other views is termed as inter-view motion parameter prediction. Furthermore, a corresponding block in a neighboring view is termed as an inter-view block and the inter-view block is located using the disparity vector derived from the depth information of current block in current picture.
FIG. 2 illustrates an example of temporal inter-view motion prediction, where the motion information of a current block (210) in a dependent view is predicted by a corresponding block (220) in an inter-view reference picture. The location of the corresponding block (220) is specified by a disparity vector (230). The motion information (222) of the corresponding block (220) is used to predict motion information (212) of the current block (210) in the current view.
View Synthesis Prediction (VSP) is a technique to remove inter-view redundancy among video signal from different viewpoints, in which synthetic signal is used as references to predict a current picture. An exemplary VSP process is illustrated in FIG. 3. VSP locates the reconstructed depth data of the reference view and uses it as virtual depth for the current PU. A technique named Neighboring Block Disparity Vector (NBDV) is used to locate the reconstructed depth data. In FIG. 3, a current prediction unit (PU) (312) in a dependent texture picture (310) is being coded. A disparity vector (330) of neighboring block (314) of the current block (312) is identified, where the disparity vector (330) points to a block (324) in the reference depth picture (320). The disparity vector (330′) is then used by the current PU (312) to location a corresponding reference depth block (322) in the reference depth picture (320). The reference depth block (322) is used as the virtual depth block for the current PU (312). Prediction signals are then generated according to a disparity vector derived from the virtual depth for each 8×8 partition of the PU. The disparity values derived from the virtual depth block are used to locate corresponding reference samples in the reference texture picture. For example, three samples in the current texture block (312) are mapped to three samples in the reference texture picture (340) according to respective disparity values as indicated by three arrows (350a-c). The mapping process is named backward warping. In addition, the warping operation may be performed at a sub-PU level precision, such as 2×2 or 4×4 blocks.
Advanced residual prediction (ARP) is another 3D coding tool used in current 3D-HEVC test model. FIG. 4 illustrates an example of advanced residual prediction (ARP) according to the current of 3D-HEVC, where the temporal residual signal in a current view is predicted by the temporal residual prediction signal in a reference view. The main procedures of ARP can be described as shown in FIG. 4, where the current prediction unit (PU 412) is a temporal prediction block using motion vector mvLX. Pictures 410 and 440 are in the current view, while pictures 420 and 450 are in the reference view. Pictures 410 and 420 correspond to two pictures with a current frame time, while pictures 440 and 450 correspond to two pictures in a reference frame time. The current block 412 is temporally predicted by temporal reference block 442 in frame 440 using motion vector mvLx. Block 422 is a corresponding block in the reference view for the current block 412. Block 422 is located from the location of block 412 according to disparity vector 430. The same motion vector mvLX is used to locate the temporal reference block (452) in the reference view corresponding to block 422. The reference residual in the reference view between block 452 and block 422 is used to predict the current residual between block 412 and block 442.
In the above 3D coding tools, i.e., IVMP, VSP and ARP, they rely on the inter-view reference picture. If the associated inter-view reference picture is not available, these 3D coding tools will not be performed correctly. Accordingly, it is desirable to develop a method to overcome the issue when the associated inter-view reference picture is not available.